Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proxy: set zero content length for default response #2459

Closed

Conversation

AlexanderYastrebov
Copy link
Member

Content-Length header prevents http.ResponseWriter from writing chunked response.

It turns out that reading empty chunked response is slow(er). In particular this change speeds up TrafficSegment tests that perform 10k requests to routes that send no body:

r50: Path("/test") && TrafficSegment(0.0, 0.5) -> status(200) -> <shunt>;
r30: Path("/test") && TrafficSegment(0.5, 0.8) -> status(201) -> <shunt>;
r20: Path("/test") && TrafficSegment(0.8, 1.0) -> status(202) -> <shunt>;

Test runtime before the change:

$ go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   9.357s

and after:

ok      github.com/zalando/skipper/predicates/traffic   4.260s

Related #1562


assert.Equal(t, http.StatusNotFound, rsp.StatusCode)
assert.Len(t, body, 0)
assert.Equal(t, "0", rsp.Header.Get("Content-Length"))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an indirect test. It is hard to test that response is not chunked because http client handles chunked encoding transparently. To test it we would need to create net connection and send/receive raw HTTP request and response.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't mind, but sounds simple enough to do:

GET / HTTP/1.1\r\n
Host: test\r\n
\r\n

@AlexanderYastrebov AlexanderYastrebov force-pushed the proxy/default-response-content-length branch from d774ac2 to ef1200f Compare July 15, 2023 23:12
@MustafaSaber
Copy link
Member

👍

Content-Length header prevents http.ResponseWriter from writing chunked response.

It turns out that reading empty chunked response is slow(er).
In particular this change speeds up TrafficSegment tests that perform
10k requests to routes that send no body:
```
r50: Path("/test") && TrafficSegment(0.0, 0.5) -> status(200) -> <shunt>;
r30: Path("/test") && TrafficSegment(0.5, 0.8) -> status(201) -> <shunt>;
r20: Path("/test") && TrafficSegment(0.8, 1.0) -> status(202) -> <shunt>;
```

Test runtime before the change:
```
$ go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   9.357s
```
and after:
```
ok      github.com/zalando/skipper/predicates/traffic   4.260s
```

Related #1562

Signed-off-by: Alexander Yastrebov <[email protected]>
@AlexanderYastrebov AlexanderYastrebov force-pushed the proxy/default-response-content-length branch from ef1200f to 4fbc3b5 Compare July 21, 2023 21:15
@szuecs
Copy link
Member

szuecs commented Aug 7, 2023

@AlexanderYastrebov I think it does not comply with https://www.rfc-editor.org/rfc/rfc9110.html#name-content-length , see for example by status code "MUST NOT".

@AlexanderYastrebov
Copy link
Member Author

@szuecs Could you quote the part you mean?

@szuecs
Copy link
Member

szuecs commented Aug 9, 2023

A server MUST NOT send a Content-Length header field in any response with a status code of 1xx (Informational) or 204 (No Content). A server MUST NOT send a Content-Length header field in any 2xx (Successful) response to a CONNECT request (Section 9.3.6).

And then there are other conditions, for example:

  1. compress filter will delete on purpose Content-Length and let the streaming do the work, if you do add Content-Length, you likely have an invalid one
  2. sed filters
  3. block filters ???
  4. Transfer-Enconding https://www.rfc-editor.org/rfc/rfc9112#name-content-length [1]

[1] A sender MUST NOT send a Content-Length header field in any message that contains a Transfer-Encoding header field.

@AlexanderYastrebov
Copy link
Member Author

AlexanderYastrebov commented Aug 9, 2023

Ok, but this sets Content-Length: 0 only for the default response of shunted route which has no body. I can add an exception for 204 status though.

@AlexanderYastrebov
Copy link
Member Author

Maybe it is not worth the effort.

@AlexanderYastrebov AlexanderYastrebov deleted the proxy/default-response-content-length branch August 9, 2023 20:49
AlexanderYastrebov added a commit that referenced this pull request Aug 9, 2023
Add `inlineContent("")` to test routes to have `Content-Length: 0`
response header that disables chunked encoding and speeds up the tests.

Before:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   8.928s
```
After:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   4.285s
```

To further speedup make test requests in parallel:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   1.554s
```

See related #2459 #1562

Signed-off-by: Alexander Yastrebov <[email protected]>
AlexanderYastrebov added a commit that referenced this pull request Aug 9, 2023
TrafficSegment tests perform 10k test requests per test case.

Add `inlineContent("")` to test routes to have `Content-Length: 0`
response header that disables chunked encoding and speeds up the tests.

Before:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   8.928s
```

After:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   4.285s
```

To further speedup make test requests in parallel:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   1.554s
```

See related #2459 #1562

Signed-off-by: Alexander Yastrebov <[email protected]>
@szuecs
Copy link
Member

szuecs commented Aug 10, 2023

@AlexanderYastrebov maybe the cases are also fine. Maybe you are right 😉

AlexanderYastrebov added a commit that referenced this pull request Aug 10, 2023
TrafficSegment tests perform 10k test requests per test case.

Add `inlineContent("")` to test routes to have `Content-Length: 0`
response header that disables chunked encoding and speeds up the tests.

Before:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   8.928s
```

After:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   4.285s
```

To further speedup make test requests in parallel:
```
go test ./predicates/traffic/ -count=1 -run=TestTrafficSegment
ok      github.com/zalando/skipper/predicates/traffic   1.554s
```

See related #2459 #1562

Signed-off-by: Alexander Yastrebov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants